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B-246474 
November  6, 1991 

The  Honorable  Edward  M.  Kennedy,  Chairman 
The  Honorable  Orrin  G.  Hatch,  Ranking  Minority  Member 
Committee  on  Labor  and  Human  Resources 
United  States  Senate 

The  Honorable  Patrick  J.  Leahy,  Chairman 
The  Honorable  Richard  G.  Lugar,  Ranking  Minority  Member 
Committee  on  Agriculture,  Nutrition  and  Forestry 
United  States  Senate 


The  Honorable  Donald  W.  Riegle,  Jr.,  Chairman 
The  Honorable  Jake  Garn,  Ranking  Minority  Member 
Committee  on  Banking,  Housing  and  Urban  Affairs 
United  States  Senate 


The  Honorable  William  D.  Ford,  Chairman 

The  Honorable  William  F.  Goodling,  Ranking  Minority  Member 

Committee  on  Education  and  Labor 

House  of  Representatives 

The  Honorable  E  (Kika)  de  la  Garza,  Chairman 
The  Honorable  E.  Thomas  Coleman,  Ranking  Minority  Member 
Committee  on  Agriculture 
House  of  Representatives 

The  Honorable  Henry  B.  Gonzalez,  Chairman 
The  Honorable  Chalmers  P.  Wylie,  Ranking  Minority  Member 
Committee  on  Banking,  Finance  and  Urban  Affairs 
House  of  Representatives 

This  report  compares  risk  ratings  done  by  bank  regulators  and  nation¬ 
ally  recognized  statistical  rating  organizations  (nrsro)  such  as  Standard 
&  Poor’s  and  Moody’s  Investors  Service.  We  are  sending  this  report 
because  your  committees  oversee  one  or  more  enterprises  in  which  the 
Treasury  Department  has  proposed  that  nrsro  ratings  qualify  an  enter¬ 
prise  for  a  regulatory  “safe  harbor.’’ 
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Various  changes  to  the  regulation  of  government-sponsored  enterprises 
have  been  proposed  by  the  Department  of  the  Treasury,  the  Congres¬ 
sional  Budget  Office,  and  us.' 

Treasury’s  regulatory  safe-harbor  provision  would  exempt  an  enterprise 
from  certain  regulatory  requirements  if  its  risk  to  the  federal  govern¬ 
ment  were  rated  as  low  as  possible  by  two  nrsros.  At  least  one  bill 
before  Congress  would  expand  the  safe-harbor  provision  to  include 
higher-risk  categories. 

In  this  report,  we  examine  the  available  evidence  to  judge  whether 
NRSRO  ratings  of  enterprises  would  provide  information  at  least  as  reli¬ 
able  and  timely  as  that  obtained  through  federal  supervision.  To  make 
this  judgment  in  the  absence  of  data  for  enterprises,  we  (1)  evaluated 
the  consistency  of  regulatory  and  nrsro  ratings  for  banks  having  both 
types  of  ratings  and  (2)  compared  the  timing  of  rating  changes  by 
NRSROS  and  federal  regulators  for  the  same  banks. 


Background 


Congress  created  government-sponsored  enterprises  to  help  make  credit 
readily  available  to  farmers,  homeowners,  and  students.  Three  enter¬ 
prises  promote  agricultural-related  lending — Farm  Credit  Banks,  Banks 
for  Cooperatives,  and  the  Federal  Agricultural  Mortgage  Corporation; 
three  promote  home  lending— Federal  Home  Loan  Banks,  the  Federal 
National  Mortgage  Association,  and  the  Federal  Home  Loan  Mortgage 
Corporation;  and  one  promotes  higher  education  lending — the  Student 
Loan  Marketing  Association. 

Enterprises  make  loans,  buy  loans  from  other  lenders,  and/or  guarantee 
financial  products,  accomplishing  their  public  purposes  through  private 
markets.  Like  any  private  financial  firm,  enterprises  are  subject  to 
financial  risks,  including  the  possibility  of  losses  arising  from  borrowers’ 
failing  to  repay  their  loans,  changes  in  interest  rates,  poor  management 
decisions,  and  unfavorable  business  conditions. 

The  federal  government  has  a  strong  interest  in  knowing  the  risks  that 
an  enterprise  takes  because  an  enterprise’s  failure  would  jeopardize  the 


'  Government-Sponsored  Enterprises;  The  Government’s  Exposure  to  Risks  (GAO/ GGD-90-97 . 

Aug.  15. 1990)  and  Government-Sponsored  Enterprises:  A  Framework  for  Limiting  the  Government's 
Exposure  to  Risks  (GAO/GGD-91-90,  May  22,  1991).  ~~  ~ 
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public  purposes  of  these  corporations  and  perhaps  lead  to  federal  assis¬ 
tance.  An  accurate  gauge  of  enterprise  risk  would  help  the  federal  gov¬ 
ernment  use  its  regulatory  authority  to  prevent  enterprise  failure  and 
possible  federal  losses. 

This  report  used  bank  data  to  compare  the  consistency  of  nrsro  and  reg¬ 
ulatory  ratings  and  the  relative  timing  of  rating  changes  made  by  each. 
Using  bank  ratings  was  imperfect  because  banks  and  enterprises  have 
different  business  activities,  although  they  face  similar  risks.  Further¬ 
more,  while  we  could  compare  a  regulator’s  risk  assessment  to  an 
NRSRO’s  assessment,  we  could  not  definitively  say  which  was  correct 
because  there  is  no  absolute  measure  of  a  financial  institution’s  risk, 
even  retrospectively.  Despite  these  limitations,  we  believe  the  experi¬ 
ence  with  regulatory  and  nrsro  bank  ratings  provides  a  useful  frame¬ 
work  with  which  to  assess  the  possible  effects  of  a  safe  harbor  in  the 
absence  of  such  ratings  for  government-sponsored  enterprises. 

To  compare  regulatory  and  nrsro  bank  ratings,  we  obtained  Moody’s 
and  Standard  &  Poor’s  credit  ratings  of  all  banks  that  had  ratings  for 
long-term  certificates  of  deposit  (CD)  greater  than  $100,000  from 
December  1985  through  December  1990.  These  banks  had  assets  ranging 
from  $100  million  to  $68  billion  in  June  1990,  with  the  majority  having 
assets  greater  than  $1  billion.  For  these  same  banks  and  time  frame,  we 
obtained  the  regulatory  assessments  of  risk.  In  total,  207  banks  had 
both  regulatory  and  nrsro  assessments  of  risk.  Of  these  207, 163  had 
Moody’s  ratings,  186  had  Standard  &  Poor’s  ratings,  and  131  had  both. 

A  detailed  description  of  our  scope  and  methodology,  together  with  limi¬ 
tations,  is  presented  in  appendix  I. 

Regulatory  and  nrsro  rating  schemes  differ  in  the  aspects  of  banks  they 
assess  and  the  language  they  use  to  rate  risk.  Bank  regulators  assess 
capital,  assets,  management,  earnings,  and  liquidity  (camel);  they  rate 
these  aspects  of  a  bank  using  a  best-to- worst  unit  scale  of  1  to  5.  In  addi¬ 
tion,  regulators  assign  an  overall  composite  rating  that  we  call  the  camel 
rating.  The  primary  purpose  of  the  camel  rating  system  is  to  help  regu¬ 
lators  identify  those  financial  institutions  that  require  special  supervi¬ 
sory  attention  and/or  warrant  a  higher  than  normal  degree  of 
supervisory  concern.  A  camel  rating  of  3  indicates  that  the  bank  is 
experiencing  some  weaknesses  ranging  from  moderately  severe  to  unsat¬ 
isfactory  but  that  failure  is  still  considered  a  remote  possibility  given 
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overall  strength  and  financial  capacity.-  Banks  with  a  camel  rating  of  5 
are  expected  to  fail  soon. 

NRSROs  rate  risk  using  letter  designations.  The  highest  ratings  (and 
lowest-risk  investments)  are  designated  “AAA”  and  “Aaa”  by  Standard 
&  Poor’s  and  Moody’s,  respectively.  In  order  of  increasing  risk,  the  scale 
moves  to  double-A,  single-A,  B,  and  C  designations.  Standard  &  Poor’s 
modifies  some  of  these  letter  designations  by  “-1-”  and  ”  signs,  while 
Moody’s  appends  a  numerical  modifier  from  1  to  3  to  further  differen¬ 
tiate  the  ratings.  In  total,  considering  the  modified  ratings,  Moody’s  and 
Standard  &  Poor’s  both  use  a  scale  with  19  risk  categories  (versus  the  5 
CAMEL  categories),  nrsro  ratings  are  opinions  about  the  credit  risk  of  a 
particular  investment.  A  rating  for  cos  is  based  on  the  nrsro’s  assess¬ 
ment  of  the  bank’s  ability  to  redeem  the  certificates  in  accordance  with 
the  terms  specified.  Standard  &  Poor’s  makes  clear  in  its  literature  that 
a  rating  is  not  a  general-purpose  evaluation  of  an  issuer  nor  a  recom¬ 
mendation  to  purchase,  sell,  or  hold  a  particular  security.  Appendix  II 
gives  a  detailed  description  of  the  three  rating  schemes  used  in  this 
report. 


Results  in  Brief 


While  CAMEL  and  NRSRO  ratings  were  fairly  closely  related  to  each  other, 
they  differed  often  enough  that  one  could  not  be  substituted  for  the 
other  with  a  great  degree  of  confidence.  However,  we  found  no  major 
differences  between  regulatory  and  nrsro  ratings  for  the  least-risky 
(triple- A-rated)  banks.  In  addition,  the  evidence  did  not  show  that  either 
regulators  or  nrsros  were  faster  at  reporting  increased  risk  at  the  least- 
risky  end  of  the  rating  scale,  in  which  a  safe-harbor  provision  would 
apply. 

Because  nrsro  and  camel  ratings  often  differed,  having  both  ratings 
would  offer  better  information  to  the  government  than  relying  on  only 
one.  If  the  evidence  available  from  banks  holds  true  for  government- 
sponsored  enterprises,  then  our  analysis  suggests  that  only  enterprises 
receiving  the  highest  possible  nrsro  ratings — triple-A  as  proposed  by 
Treasury — should  be  eligible  for  exemption  from  regulatory 
requirements. 


^Even  though  a  CAMF^L  rating  of  3  is  at  the  midpoint  in  the  l-to-.S  CAMEL  scale,  it  d(x;s  not  represent 
an  average-risk  bank.  Flighty  percent  of  CAMEL  ratings  were  1  or  2  for  the  banks  we  examined. 
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Private  Agency 
Ratings  Do  Not 
Substitute  Well  for 
Regulatory  Ratings  of 
Banks 


Table  1:  Percentage  of  CAMEL  High-Risk 
Observations  That  NRSROs  Rated  as 
Low-Risk 


While  there  was  a  positive  association  between  camel  and  nrsro  bank 
ratings,  they  differed  often  enough  that  one  could  not  be  readily  substi¬ 
tuted  for  the  other.  We  used  two  analytical  techniques  to  evaluate  the 
association  between  camel  and  nrsro  risk  assessments — tables  of  fre¬ 
quency  distributions  and  correlation  analysis. 

One  way  we  analyzed  the  difference  between  regulatory  and  nrsro  rat¬ 
ings  was  to  look  at  cases  of  disagreements.  A  disagreement  suggests  that 
the  regulator  or  the  nrsro  misclassified  the  bank’s  risk.  Table  1  displays 
the  cases  when  an  nrsro  rating  was  low-risk  while  the  regulatory  camel 
rating  was  high-risk. 


CAMEL  rating 

. . . 

Moody’s 

3 

4 

5 

Total 

Aaa 

0.0% 

0.0% 

0.0% 

0.0% 

Aa1 ,  Aa2,  Aa3 

8.6 

0.3 

0.0 

8.9 

A1,  A2 

26.0 

3.3 

0.0 

29.3 

Total 

34.6% 

3.6% 

0.0% 

38.2% 

Standard  &  Poor’s  | 

AAA 

0.0 

0.0 

0.0 

0.0 

AA+,  AA,  AA- 

8.5 

2.9 

0,0 

11.4 

A+,  A,  A- 

35.7 

7.9 

0,0 

43.8 

Total 

44.2% 

10.8% 

0.0% 

55.0% 

Note:  For  this  table,  we  defined  CAMEL  1  and  2  ratings  as  low-risk.  Since  these  were  approximately  80 
percent  of  CAMEL  ratings  (see  appendix  III),  we  defined  approximately  80  percent  of  Moody's  and  Stan¬ 
dard  &  Poor's  ratings  as  low-risk  and  approximately  20  percent  of  their  ratings  as  high-risk.  This  defini¬ 
tion  classified  the  first  seven  Standard  &  Poor's  ratings  but  only  the  first  six  Moody's  ratings  as  low-risk. 
Were  Moody's  A3  rating  included,  then  89  percent  of  Moody's  ratings  would  be  classified  as  low-risk 
and  the  total  disagreement  between  Moody's  and  CAMEL  ratings  would  change  from  38.2  percent  to 
57.6  percent  in  the  table. 

Source:  GAO  analysis  based  on  ratings  supplied  by  Moody's,  Standard  &  Poor's,  and  federal  bank 
regulators. 

As  table  1  shows,  there  were  no  cases  where  Moody’s  or  Standard  & 
Poor’s  gave  their  best  credit  rating — Aaa  or  AAA,  respectively — when 
the  CAMEL  rating  indicated  that  the  bank  was  exhibiting  weaknesses.  If 
these  results  from  banks  are  assumed  to  be  consistent  with  expected 
experience  for  enterprises,  then  a  safe-harbor  proposal  limited  to  enter¬ 
prises  receiving  two  triple-A  ratings  would  be  unlikely  to  result  in  a  mis- 
classification  of  risk.  Were  the  safe-harbor  proposal  to  incorporate 
rating  categories  below  this  highest  level,  then  the  possibility  of  misclas- 
sification  would  be  more  likely. 
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Table  2:  Correlaiion  of  CAMEL  and 
NRSRO  Ratings 


Next,  we  used  correlation  analysis  to  indicate  how  well  krsro  ratings 
matched  aA.MEL  ratings.  Although  they  were  positively  related,  the 
highest  correlation — of  .67 — indicated  that  the  two  risk  assessments 
differed  a  considerable  part  of  the  time.  Moody’s  ratings  were  better 
correlated  with  camel  than  were  Standard  &  Poor’s.  Moody’s  and  Stan¬ 
dard  &  Poor’s  were  better  correlated  with  each  other  than  either  was 
correlated  with  camel.  Table  2  summarizes  the  results  of  our  correlation 
analyses. 


Ratings  correlated 

Correlation  coefficient 

Standard  &  Poor’s  and  CAMEL 

.53 

Moody’s  and  CAMEL 

,67 

Moody’s  and  Standard  &  Poor’s 

.86 

These  results  suggest  that  nrsros  and  bank  regulators  do  not  have  a 
common  view  of  the  risks  of  a  bank’s  activities.  One  difference  appears 
to  involve  the  different  purposes  of  regulatory  and  nrsro  ratings,  camel 
ratings  identify  and  distinguish  among  banks  that  need  special  supervi¬ 
sory  attention.  The  high-risk  end  of  the  scale  is  very  important  in 
accomplishing  this  purpose  because  high-risk  banks  may  involve  deposit 
insurance  losses.  However,  the  nrsros  have  a  different  view  of  the  high- 
risk  end  of  the  scale.  In  assessing  risk,  nrsros  typically  consider  the 
effects  of  regulatory  intervention  in  protecting  the  investors  holding 
CDS.  One  Moody’s  official  explained  that,  when  Moody’s  determines  that 
the  Federal  Deposit  Insurance  Corporation  (fdic)  is  likely  to  provide 
assistance  to  a  bank  that  in  effect  will  protect  uninsured  depositors 
from  losses,  Moody’s  will  generally  not  assign  a  lower  rating  than  Ba3. 
Standard  &  Poor’s  suggested  in  its  literature  that  CDs  are  assigned  lower 
ratings  even  when  regulatory  support  can  be  expected.  However,  Stan¬ 
dard  &  Poor’s  did  not  use  the  highest-risk  end  of  its  rating  scale  for  any 
of  the  banks  that  we  reviewed. 


Neither  Rating 
Agencies  Nor 
Regulators 

Consistently  Detected 
Risk  Changes  Earlier 


Neither  the  regulators  nor  the  nrsros  consistently  reported  changes  in 
risk  earlier  than  the  other.  We  analyzed  the  timing  of  rating  changes 
using  two  techniques — (1)  correlation  analyses  that  varied  the  dates  to 
analyze  patterns  of  early  or  late  reporting  of  rating  changes  and  (2) 
event  history  analysis. 

Correlation  coefficients  were  computed  using  an  nrsro  rating  at  a  partic¬ 
ular  date  matched  with  the  camel  rating  at  the  same  date,  at  earlier 
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dates,  and  at  later  dates.  Our  expectation  was  that  if  camel  ratings 
changed  assessments  of  risk  consistently  before  the  nrsros,  then  the 
highest  correlation  coefficients  would  occur  when  the  nrsro  ratings  at  a 
particular  month  were  correlated  with  camel  ratings  at  some  consist¬ 
ently  earlier  date. 

We  found  that  correlation  coefficients  were  not  significantly  different 
when  the  series  were  matched  at  the  same  date  or  when  one  series  was 
matched  with  nearby  earlier  and  later  dates  of  another.  Our  analysis 
and  results  are  explained  further  in  appendix  IV. 

The  second,  more  definitive  analysis  that  we  did  on  the  question  of  rela¬ 
tive  timing  of  rating  changes  used  a  technique  called  event  history  anal¬ 
ysis.  Using  statistical  tests,  we  determined  whether  nrsros  or  the 
regulators  were  faster  in  recognizing  risk  increases.  We  did  this  by 
looking  at  cases  in  which  both  initially  agreed  on  the  level  of  risk;  then 
one  said  risk  had  increased,  but  the  other  did  not;  then  both  eventually 
agreed  that  risk  had  increased.  We  found  that  Moody’s  was  neither 
faster  nor  slower  than  the  regulators  in  detecting  increases  in  risk  at  the 
lo west-risk  end  of  the  rating  scale.® 


Conclusions 


Evidence  from  banks  indicated  that  nrsro  and  camel  ratings  generally 
agreed  but  differed  often  enough  that  one  could  not  be  substituted  for 
the  other  with  a  great  degree  of  confidence.  Evidence  from  banks  did 
not  suggest  that  either  regulators  or  nrsros  were  faster  at  detecting 
increases  in  risk  at  the  lower-risk  end  of  the  rating  scale.  Of  particular 
interest  to  the  safe-harbor  concept,  we  found  no  case  in  which  an  nrsro 
rated  a  bank  triple-A  but  regulators  found  the  bank’s  condition  to  cause 
supervisory  concern.  Banks  receiving  double-A  ratings  or  lower  were 
sometimes  the  cause  of  supervisory  concern. 


Matters  for 
Consideration  by 
Congress 


If  the  experience  from  banks  provides  a  reasonable  basis  for  assessing 
how  an  enterprise  safe-harbor  would  work,  then  Congress  may  want  to 
restrict  any  regulatory  safe-harbor  to  triple-A  rated  enterprises. 


®Due  to  time  constraints,  the  event  history  analysis  was  done  using  only  Moody’s  data. 
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Agency  Comments 


Officials  from  the  Office  of  the  Comptroller  of  the  Currency  (occ),  the 
Federal  Reserve  Board  of  Governors,  fdic,  Moody’s  Investors  Service, 
and  Standard  &  Poor’s  reviewed  a  draft  of  this  report.  These  officials 
generally  agreed  with  the  contents  of  this  report.  We  have  incorporated 
their  clarifications  and  comments  where  appropriate. 


We  are  providing  copies  of  this  report  to  federal  bank  regulators,  execu¬ 
tive  branch  agencies,  government-sponsored  enterprises,  Moody’s  Inves¬ 
tors  Service,  Standard  &  Poor’s,  and  other  interested  parties. 

The  major  contributors  to  this  report  are  listed  in  appendix  VI.  Please 
contact  me  at  (202)  275-8678  if  you  or  your  staff  have  any  questions 
concerning  the  report. 


Craig  A.  Simmons 
Director,  Financial  Institutions 
and  Markets  Issues 
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Objective 


In  this  report,  v^^e  examined  the  available  evidence  to  judge  whether 
NRSRO  ratings  of  government-sponsored  enterprises  would  provide  infor¬ 
mation  at  least  as  reliable  and  timely  as  that  obtained  through  federal 
supervision.  To  make  this  judgment  in  the  absence  of  data  for  enter¬ 
prises,  we  (1)  evaluated  the  consistency  of  regulatory  and  nrsro  ratings 
for  banks  having  both  types  of  ratings  and  (2)  compared  the  timing  of 
rating  changes  by  nrsros  and  federal  regulators  for  the  same  banks. 


Scope 


We  obtained  bank  ratings  from  Moody’s  and  Standard  &  Poor’s  as  far 
back  as  they  maintained  the  data  in  computer-accessible  form — from 
December  1985  through  December  1990.  We  then  obtained  camel  ratings 
for  these  banks  from  each  bank’s  primary  regulator — the  occ,  the  Fed¬ 
eral  Reserve  Board  of  Governors,  or  the  epic.' 

This  gave  us  a  universe  of  207  banks  that  had  ratings  both  from  a  regu¬ 
lator  and  at  least  one  nrsro.  These  banks  had  assets  ranging  in  size  from 
$100  million  to  $68  billion,  with  the  majority  having  assets  greater  than 
$  1  billion.  Since  our  analyses  compared  the  ratings  from  only  one  nrsro 
at  a  time  to  the  corresponding  camel  ratings,  the  effective  universe  was 
153  banks  for  analyses  using  Moody’s  data  and  185  banks  for  analyses 
using  Standard  &  Poor’s  data.  However,  for  various  reasons,  not  all 
banks  had  ratings  spanning  the  entire  time  period  from  December  1985 
through  December  1990.  Moody’s  or  Standard  &  Poor’s  may  have 
started  rating  some  banks  after  December  1985;  some  banks  may  have 
started  operations  after  this  date;  and  others  may  have  ceased  opera¬ 
tions  before  December  1990  due  to  insolvency  or  a  purchase  by  or 
merger  with  another  bank. 

Since  they  are  similar  measures  of  risk,  we  chose  to  use  Moody’s  and 
Standard  &  Poor’s  ratings  for  bank  long-term  cos  greater  than  $100,000 
to  compare  to  the  bank  regulators’  camel  ratings.  Because  CD  amounts 
greater  than  $100,000  are  uninsured,  nrsro  ratings  measured  the  risk 
uninsured  depositors  have  of  incurring  losses  if  the  bank  fails.  Similarly, 
camel  ratings  measured  the  risk  to  the  federal  deposit  insurance  fund 
that  it  would  have  to  cover  losses  in  the  event  of  a  bank  failure. 

Regulators  rate  bank  risk  using  a  unit  scale  from  1  (low)  to  5  (high). 
Moody’s  and  Standard  &  Poor’s  use  letter  grades  to  rate  the  risk  of  long¬ 
term  financial  instruments.  The  lowest-risk  grades  are  designated 


‘The  bank  regulators  supplied  confidential  bank  rating  data  to  us  with  the  condition  that  individual 
bank  ratings  not  be  revealed. 
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Methodology 


“AAA”  and  ‘  Aaa”  by  Standard  &  Poor’s  and  Moody’s,  respectively.  In 
order  of  increasing  risk,  the  scale  moves  to  double-A,  single-A,  B,  and  C 
designations.  Standard  &  Poor’s  modifies  some  of  these  letter  designa¬ 
tions  by  and  signs,  while  Moody’s  appends  a  numerical  modifier 
from  1  to  3  to  further  differentiate  the  ratings.  In  total,  considering  the 
modified  ratings,  Moody’s  and  Standard  &  Poor’s  both  use  a  scale  with 
19  risk  categories.  Appendix  II  gives  a  detailed  description  of  the  three 
rating  schemes. 

Only  13  of  Moody’s  ratings  (from  Aaa  to  Ba3)  and  only  16  of  Standard  & 
Poor’s  ratings  (from  AAA  to  B-)  appeared  in  their  ratings  of  banks.  We 
assigned  a  unit  scale  from  1  (low)  to  13  (high)  to  Moody’s  ratings  and  1 
(low)  to  16  (high)  to  Standard  &  Poor’s  ratings  to  compare  them  to  the  1- 
to-5  CAMEL  rating  scale. 

Because  nrsro  and  camel  ratings  often  remained  unchanged  for  long 
periods — months,  sometimes  years — we  transformed  the  data  into 
monthly  observations  by  extending  the  rating  of  a  particular  day  to  the 
month  in  which  it  was  made  and  all  subsequent  months  until  the  occur¬ 
rence  of  a  new  rating.  When  there  was  more  than  one  rating  given  in  a 
month,  we  assigned  the  rating  in  effect  on  the  last  day  of  the  month  to 
that  entire  month,  nrsro  ratings  were  dated  when  the  ratings  became 
public,  but  the  camel  ratings  were  dated  either  (1)  the  date  of  the  finan¬ 
cial  data  on  which  the  bank  examination  was  based  or  (2)  the  date  the 
examination  began.  Bank  regulators  told  us  that  for  banks  of  the  sizes  in 
our  sample,  ratings  are  actually  assigned  an  average  of  3  to  4  months 
after  the  recorded  dates.  However,  they  cautioned  that  this  rough  esti¬ 
mate  involved  a  large  variance  (i.e.,  individual  cases  may  be  quicker  or 
later).  Consequently,  we  analyzed  the  data  first  as  originally  supplied  to 
us  by  the  regulators  and  then,  with  regulatory  dates  lagged  by  3 
months.2 


We  used  two  techniques  to  evaluate  the  consistency  of  nrsro  and  camel 
ratings.  Correlation  analysis  was  used  to  indicate  how  well  changes  in 
CAMEL  and  NRSRO  ratings  corresponded.  Both  camel  and  nrsro  ratings  are 
inherently  rank-ordered,  so  we  used  a  Pearson^  correlation  coefficient  to 


^However,  since  the  regulators  found  it  difficult  to  supply  us  with  an  average  figure  in  which  they 
felt  a  great  degree  of  confidence,  we  performed  some  additional  analysis  and  reported  when  our 
results  were  sensitive  to  the  assumed  adjustment. 

^The  Spearman  rank  correlation  coefficient  is  the  Pearson  correlation  coefficient  computed  between 
the  ranked  values  of  two  series.  Both  CAMEL  and  NRSRO  ratings  are  rank-ordered,  so  we  used  the 
Pearson  correlation  coefficient. 
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measure  the  association  between  cami-:l  and  nrsro  ratings^  We  also  used 
a  two-dimensional  frequency  table  to  examine  the  relationship  between 
the  ratings  at  varying  risk  levels.  This  table  allowed  us  to  calculate  the 
percentage  of  observations  that  disagreed,  suggesting  one  or  both  rat¬ 
ings  might  have  misclassified  risk  (e.g.,  where  Moody’s  or  Standard  & 
Poor’s  classified  risk  to  be  low  when  regulators  classified  it  to  be  high). 

We  also  used  two  techniques  to  evaluate  the  relative  timing  of  camel  and 
NRSRO  rating  changes.  Correlation  coefficients  were  computed  using  the 
value  of  an  nrsro  rating  at  a  particular  date  matched  with  the  camel 
rating  at  the  same  date,  at  earlier  dates,  and  at  later  dates.  Our  expecta¬ 
tion  was  that  if  one  rating  series  (for  example,  camel)  changed  assess¬ 
ments  of  risk  consistently  before  the  other,  then  the  highest  correlation 
coefficients  would  occur  when  nrsro  ratings  were  correlated  with  camel 
ratings  at  some  consistently  earlier  date.  Again,  a  Pearson  correlation 
coefficient  was  the  statistic  used  to  measure  this  association."’ 

Because  the  correlation  analysis  technique  did  not  allow  us  to  differen¬ 
tiate  among  risk  levels,  we  also  used  event  history  analysis  to  determine 
if  one  rating  series  led  the  other  in  detecting  risk  increases  at  particular 
levels  of  risk.  We  specifically  focused  on  the  case  where  risk  was  low 
because  this  is  where  the  safe-harbor  proposal  would  apply.  We  used 
chi-square  tests  to  assess  whether  either  camel  leads  nrsro  or  nrsro 
leads  CAMEL  in  assigning  higher  risk  ratings  to  banks.  Specifically,  we 
concluded  that  ca.mel  leads  nrsro  when  the  following  two  hypotheses 
were  supported  based  on  chi-square  tests: 

(1)  After  the  two  ratings  agree  that  a  bank  was  low-risk  in  a  month, 
CAMEL  was  more  likely  than  the  nrsro’s  ratings  to  report  the  bank  as 
“high-risk”  the  following  month. 

(2)  When  the  two  ratings  disagreed  in  a  month  (camel  “high-risk”  and 
nrsro  “low-risk”),  the  two  ratings  were  more  likely  to  agree  that  the 
bank  was  “high-risk”  in  the  following  month  than  to  agree  that  the  bank 
was  “low-risk”  in  the  following  month. 

The  first  hypothesis  asserts  that  camel  leads  and  the  second  asserts  that 
nrsro  follows.  Only  when  the  data  supported  both  camel  leads  and 


‘*In  this  case,  monthly  comparative  observations  were  weighted  equally. 

'""In  this  case,  each  bank  was  weighted  equally  when  computing  the  statistic.  For  example,  a  bank 
with  20  months  of  obseiwations  w’ould  have  been  given  equal  weight  as  a  bank  with  40  months  of 
obseiwations  in  computing  an  average  correlation  coefficient. 
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NRSRO  follows  did  we  infer  that  camel  leads  the  nrsro  in  reporting 
increases  in  risk.  An  analogous  pair  of  chi-square  tests  was  used  to 
assess  whether  the  nrsro  leads  camel. 

The  use  of  the  frequency  table  to  identify  rating  disagreement  and  the 
event  history  analyses  required  us  to  define  a  particular  association  of 
CAMEL  and  nrsro  ratings.  Since  there  was  no  intrinsically  correct  way  to 
do  this,  we  used  a  number  of  associations  by  (1)  using  definitions  of  the 
rating  schemes  as  guidelines  (see  app.  II),  (2)  arranging  nrsro  ratings  so 
that  they  had  a  similar  proportion  of  more-  and  less-risky  ratings  as 
CAMEL  for  the  banks  reviewed,  and  (3)  using  regression  analysis.  We  also 
used  associations  that  fell  within  the  range  of  those  resulting  from  these 
methods. 


A  key  assumption  of  our  analysis  was  that  the  way  various  factors 
influenced  a  bank’s  nrsro  and  camel  rating  remained  constant 
throughout  the  period  analyzed.  In  addition,  our  research  had  several 
limitations.  First,  the  two  rating  schemes  do  not  measure  exactly  the 
same  thing.  Although  both  measure  risk,  they  do  so  in  different  ways 
and  for  different  purposes.  (See  app.  II  for  more  information  on  the  pur¬ 
poses  and  definitions  of  rating  schemes.)  However,  since  both  ratings 
indicate  a  bank’s  risk  level  and  should  move  in  the  same  direction  at  the 
same  time,  we  concluded  that  comparisons  could  provide  useful 
information. 

Second,  our  ability  to  make  judgments  at  the  highest-risk  levels  was  lim¬ 
ited  because  nrsros  reported  no  bank  cos  at  their  highest-risk  levels. 
NRSROs  consider  the  federal  practice  of  protecting  uninsured  depositors 
from  losses  resulting  from  large  bank  failures  when  they  rate  cos.  To 
the  degree  that  nrsros  do  not  use  the  full  extent  of  their  rating  scale 
because  they  expect  that  bank  depositors  will  be  protected  from  losses, 
the  comparisons  between  camel  and  nrsro  ratings  could  be  biased  at  the 
highest-risk  end  of  the  scale.  However,  we  were  most  interested  in  the 
sensitivity  of  regulators’  and  nrsros’  ability  to  detect  increases  in  risk  at 
the  less-risky  end  of  the  ratings  scale  where  Treasury’s  “safe-harbor” 
proposal  would  apply.  Thus,  possible  bias  of  nrsro  ratings  at  the  riskier 
end  of  the  rating  scale  did  not  rule  out  using  the  ratings  for  our  purpose. 

Another  limitation  was  our  inability  to  definitively  answer  the  question 
of  how  well  the  rating  systems  perform  in  assessing  risk  and  in  recog¬ 
nizing  changes  in  risk  because  there  is  no  absolute  measure  of  this  risk. 
To  overcome  this  limitation,  we  used  several  techniques  to  compare  the 


Assumptions  and 
Limitations  of  the 
Data  and  Analysis 
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ratings  and  several  plausible  ways  to  establish  a  correspondence 
between  them;  these  techniques  enabled  us  to  note  if  and  how  the 
results  were  affected  by  our  choice  of  analysis  technique  and  correspon¬ 
dence  scheme. 

Our  work  was  undertaken  between  December  1990  and  October  1991  in 
accordance  wth  generally  accepted  government  auditing  standards. 
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This  appendix  discusses  the  three  rating  schemes  used  in  this  report: 

•  the  uniform  financial  institutions  rating  system,  or  camel,  used  by  bank 
regulators; 

•  Moody’s  Investors  Service  long-term  CD  ratings;  and 

•  Standard  &  Poor’s  long-term  CD  ratings. 


CAMEL 


The  CAMEL  rating  system  provides  a  general  framework  for  evaluating 
and  assimilating  all  significant  financial,  operational,  and  compliance 
factors  in  order  to  assign  a  summary  or  composite  rating  to  each  feder¬ 
ally  regulated  commercial  bank,  savings  and  loan  association,  mutual 
savings  bank,  and  credit  union.*  The  purpose  of  the  rating  system  is  to 
reflect  in  a  comprehensive  and  uniform  fashion  an  institution’s  financial 
condition,  compliance  with  laws  and  regulations,  and  overall  operating 
soundness.  Financial  institution  regulators  first  assess  capital,  assets, 
management,  earnings,  and  liquidity.  Each  of  these  components  is  rated 
individually  according  to  a  best-to- worst  unit  scale  of  1  to  5.  The  regula¬ 
tors  then  assign  an  overall  composite  rating  using  the  same  scale. 

The  primary  purpose  of  the  composite  rating  is  to  help  identify  those 
institutions  that  require  special  supervisory  attention  and/or  warrant 
more  than  normal  supervisory  concern.  In  an  effort  to  accomplish  this 
objective,  the  rating  system  identifies  certain  institutions  whose  finan¬ 
cial,  operational,  or  managerial  weaknesses  are  so  severe  as  to  pose  a 
serious  threat  to  continued  financial  viability.  These  institutions  receive, 
depending  upon  degree  of  risk  and  supervisory  concern,  a  rating  of  “4” 
or  “5.”  Such  institutions  are  generally  characterized  by  unsafe, 
unsound,  or  other  seriously  unsatisfactory  conditions  and  carry  a  rela¬ 
tively  high  possibility  of  failure  or  insolvency. 

The  CAMEL  rating  system  also  identifies  a  category  of  institutions  that 
have  some  combination  of  financial  or  compliance  deficiencies  that, 
while  posing  little  or  no  threat  to  financial  viability  under  present  cir¬ 
cumstances,  do  warrant  more  than  normal  supervisory  concern.  These 
institutions  are  not  deemed  to  present  a  significant  risk  of  failure  or  loss 
or  hardship  to  depositors,  borrowers,  or  the  public,  but  do  require  a 
higher  than  normal  level  of  supervision.  Institutions  that  warrant  some 
supervisory  concern  but  do  not  entail  a  relatively  high  possibility  of 
failure  or  insolvency  are  generally  rated  “3.” 


'information  in  thi.s  section  was  largely  excerpted  from  the  “Uniform  Financial  Institutions  Rating 
System”  recommended  by  the  Federal  Financial  Institutions  Examination  Council,  Nov.  13,  1979. 
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Composite  ratings  are  defined  and  distinguished  as  follows: 

»  Composite  1  institutions  are  basically  sound  in  every  respect;  any  crit¬ 
ical  findings  or  comments  are  of  a  minor  nature  and  can  be  handled  in  a 
routine  manner.  Such  institutions  are  resistant  to  external  economic  and 
financial  disturbances  and  more  capable  of  withstanding  the  vagaries  of 
business  conditions  than  institutions  with  lower  ratings. 

•  Composite  2  institutions  are  fundamentally  sound  but  may  reflect 
modest  weaknesses  correctable  in  the  normal  course  of  business.  The 
nature  and  severity  of  deficiencies  are  not  considered  material.  There¬ 
fore,  such  institutions  are  stable  and  also  able  to  withstand  business 
fluctuations  quite  well. 

»  Composite  3  institutions  exhibit  a  combination  of  financial,  operational, 
or  compliance  weaknesses  ranging  from  moderately  severe  to  unsatis¬ 
factory.  When  weaknesses  relate  to  financial  condition,  such  institutions 
may  be  vulnerable  to  the  onset  of  adverse  business  conditions  and  could 
easily  deteriorate  if  concerted  action  is  not  effective  in  correcting  the 
areas  of  weakness.  Institutions  that  are  in  significant  noncompliance 
with  laws  and  regulations  may  also  be  accorded  this  rating.  Generally, 
these  institutions  give  cause  for  supervisory  concern  and  require  more 
than  normal  supervision  to  address  deficiencies.  Overall  strength  and 
financial  capacity,  however,  are  still  such  as  to  make  failure  only  a 
remote  possibility. 

»  Composite  4  institutions  have  an  immoderate  volume  of  serious  finan¬ 
cial  weaknesses  or  a  combination  of  other  conditions  that  are  unsatisfac¬ 
tory.  Major  and  serious  problems  or  unsafe  and  unsound  conditions  may 
exist  that  are  not  being  satisfactorily  addressed  or  resolved.  Unless 
effective  action  is  taken  to  correct  these  conditions,  they  could  reason¬ 
ably  develop  into  a  situation  that  could  impair  future  viability,  consti¬ 
tute  a  threat  to  the  interests  of  depositors,  and/or  pose  a  potential  for 
disbursement  of  funds  by  the  insuring  agency.  A  higher  potential  for 
failure  is  present  but  is  not  yet  imminent  or  pronounced. 

•  Composite  5  institutions  have  an  extremely  high  immediate  or  near-term 
probability  of  failure.  The  volume  and  severity  of  weaknesses  or  unsafe 
and  unsound  conditions  are  so  critical  as  to  require  urgent  aid  from 
stockholders  or  other  public  or  private  sources  of  financial  assistance.  In 
the  absence  of  urgent  and  decisive  corrective  measures,  these  situations 
will  likely  require  liquidation  and  the  payoff  of  depositors;  disburse¬ 
ment  of  insurance  funds  to  insured  depositors;  or  some  form  of  emer¬ 
gency  assistance,  merger,  or  acquisition. 
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Moody’s  Investors  Service  used  the  following  ratings  for  senior  bank 
obligations  having  maturities  longer  than  1  year.^ 

•  Aaa  obligations  are  of  the  best  quality  and  carry  the  smallest  degree  of 
investment  risk.  Interest  payments  are  protected  by  a  large  or  excep¬ 
tionally  stable  margin  and  principal  is  secure.  While  the  various  protec¬ 
tive  elements  are  likely  to  change,  such  changes  as  can  be  visualized  are 
most  unlikely  to  impair  the  fundamentally  strong  position  of  such 
issues. 

•  Aa  obligations  are  judged  to  be  of  high  quality  by  all  standards. 

Together  with  the  Aaa  group  they  comprise  what  are  generally  known 
as  high-grade  bonds.  They  are  rated  lower  than  the  best  obligations 
because  margins  of  protection  may  not  be  as  large  as  in  Aaa  securities, 
fluctuation  of  protective  elements  may  be  of  greater  amplitude,  or  there 
may  be  other  elements  present  that  make  the  long-term  risk  appear 
somewhat  larger  than  the  Aaa  securities.  - 

•  A  obligations  possess  many  favorable  investment  attributes  and  are  to 
be  considered  as  upper-medium-grade  obligations.  Factors  giving 
security  to  principal  and  interest  are  considered  adequate,  but  elements 
may  be  present  that  suggest  a  susceptibility  to  impairment  some  time  in 
the  future. 

•  obligations  are  considered  medium-grade  (i.e.,  they  are  neither 
highly  protected  nor  poorly  secured).  Interest  payments  and  principal 
security  appear  adequate  for  the  present  but  certain  protective  elements 
may  be  lacking  or  may  be  characteristically  unreliable  over  any  length 
of  time.  Such  obligations  lack  outstanding  investment  characteristics 
and  in  fact  have  speculative  characteristics. 

•  Ba  obligations  are  judged  to  have  speculative  elements;  their  future 
cannot  be  considered  well-assured.  Often  the  protection  of  interest  and 
principal  payments  may  be  very  moderate,  and,  thereby,  not  well  safe¬ 
guarded  during  both  good  and  bad  times  over  the  long  term.  Uncertainty 
of  position  characterizes  bonds  in  this  class. 

•  B  obligations  lack  characteristics  of  the  desirable  investment.  Assurance 
of  interest  and  principal  payments  or  of  maintenance  of  other  terms  of 
the  contract  over  any  long  period  of  time  may  be  small. 

•  obligations  are  of  poor  standing.  Such  issues  may  be  in  default  or 
may  have  elements  of  danger  with  respect  to  principal  or  interest. 

•  Ca  obligations  are  highly  speculative.  Such  issues  are  often  in  default  or 
have  other  marked  shortcomings. 


-Largely  excel  pted  from  Moody  s  Investors  Service,  Credit  Opinions:  Financial  Institutions,  Dec.  1989. 
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.  C  obligations  are  the  lowest-rated  class  and  can  be  regarded  as  having 
^tremely  poor  prospects  of  ever  attaining  any  real  investment 
standing. 

Moody’s  applies  numerical  modifiers  1 ,  2,  and  3  in  each  generic  rating 
classification  from  Aa  to  B.  The  modifier  of  1  indicates  that  the  invest¬ 
ment  is  in  the  higher-credit  (lower-risk)  end  of  its  generic  rating  cate¬ 
gory;  the  modifier  2  indicates  a  mid-range  ranking;  and  the  modifier  3 
indicates  that  the  ranking  is  in  the  lower-credit  (higher-risk)  end  of  the 
generic  category. 


Standard  &  Poor’s 
Bank  CD  Rating 
System 


Standard  &  Poor’s  provides  investors  with  an  evaluation  of  the 
creditworthiness  of  cos  issued  by  U.S.  banks,  savings  and  loan  institu¬ 
tions,  and  foreign  banks.^  The  rating  judgment  is  a  composite  of  two 
assessments.  Fundamental  credit  analysis  on  each  company  is  per¬ 
formed  using  quantitative  analysis  to  determine  an  institution’s  finan¬ 
cial  position,  including  portfolio  quality,  liquidity,  profitability,  and 
capital  adequacy.  The  second  assessment  focuses  on  subjective  factors, 
such  as  management  depth  and  quality,  risk  profile,  business  aggres¬ 
siveness,  and  regulatory  support.  Where  the  institution’s  financial 
strength  is  questionable,  cos  are  assigned  lower  ratings  even  when  regu¬ 
latory  support  can  be  expected.  Standard  &  Poor’s  assesses  the  issuer’s 
ability  to  redeem  the  CD  in  accordance  with  the  terms  specified. 

The  rating  definitions  and  their  respective  meanings  follow; 


Long-Term  Investment 
Grades 


AAA  represents  the  highest  degree  of  safety.  These  issues  have  an  over¬ 
whelming  repayment  capacity. 

AA  represents  a  very  high  degree  of  safety  and  capacity  for  repayment. 
These  issues  differ  from  higher-rated  issues  only  to  a  small  degree. 

A  represents  a  strong  degree  of  safety  and  capacity  for  repayment,  but 
these  issues  are  somewhat  more  susceptible  in  the  long  term  to  adverse 
economic  conditions  than  those  rated  in  higher  categories. 

BBB  represents  a  satisfactory  degree  of  safety  and  capacity  for  repay- 
ment,  but  these  issues  are  more  vulnerable  to  adverse  economic  condi¬ 
tions  or  changing  circumstances  than  higher-rated  issues. 


■■^Largely  excerpted  from  Standard  &  Poor’s,  “Bank  and  Savings  &  Loan  CD  Ratings,”  May  1990 
Supplement. 
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Long-Term  Speculative 
Grades 


•  M  represents  less  near-term  vulnerability  to  default  than  other  specula¬ 
tive  issues.  However,  the  issues  face  major  ongoing  uncertainties  or 
exposures  to  adverse  economic  or  financial  conditions,  thus  threatening 
capacity  to  meet  interest  or  principal  payments  on  a  timely  basis. 

•  B  represents  greater  vulnerability  to  default  but  currently  has  the 
capacity  to  meet  interest  and  principal  repayments.  Adverse  business, 
financial,  or  economic  conditions  will  likely  impair  capacity  to  pay 
interest  and  repay  principal. 

•  CCC  represents  issues  having  currently  identifiable  vulnerability  to 
default.  Adverse  business,  financial,  or  economic  developments  would 
render  repayment  capacity  unlikely. 

Standard  &  Poor’s  ratings  from  AA  to  CCC  may  be  modified  by  a  plus  or 
minus  sign,  reflecting  the  relative  standing  within  the  major  rating  cate¬ 
gories.  The  “-I-”  modifier  indicates  that  the  investment  is  at  the  higher- 
credit  (lower-risk)  end  of  the  major  rating  category;  and  the  modifier 
indicates  that  the  investment  is  in  the  lower-credit  (higher-risk)  end  of 
the  major  rating  category. 
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CAMEL  ratinas 

Moody’s 

1 

2 

3 

4 

5 

Total 

Aaa 

3.11 

1.40 

0 

0 

0 

4.51 

Aa1 

3.60 

3.22 

.16 

0 

0 

6.98 

Aa2 

5.32 

10.17 

,60 

.01 

0 

16.11 

Aa3 

3.94 

20,02 

1.06 

,06 

0 

25.07 

A1 

2.50" 

13,14 

3,58 

,10 

0 

19.32 

A2 

1.30 

6.68 

1.91 

.60 

0 

10.49 

A3 

.04 

2.25 

3.21 

.88 

.01 

6.41 

Baa1 

.33 

,94 

2.77 

1.03 

0 

5.06 

Baa2 

0 

.58 

1.47 

.53 

,16 

2.74 

Baa3 

0 

.04 

.40 

.41 

.23 

1.08 

Ba1 

0 

.27 

.17 

.53 

20 

1.17 

Ba2 

0 

0 

0 

.21 

.26 

.47 

Ba3 

0 

0 

,03 

0 

.57 

.60 

Total 

20.13 

58.72 

15.35 

4.37 

1.43 

100.00 

Standard  &  Poor’s 

AAA 

1.30 

0 

0 

0 

0 

1.30 

AA+ 

1.74 

2.48 

.23 

0 

0 

4.45 

AA 

4.35 

7.72 

.63 

.28 

0 

12,97 

AA- 

3.94 

4.90 

.63 

,23 

0 

9.70 

A+ 

5.84 

10.31 

1.28 

,13 

0 

17.56 

A 

4.85 

16.70 

1.87 

,51 

0 

23.94 

A- 

1.84 

9.68 

3.11 

.74 

0 

15.37 

BBB+ 

1.35 

3.40 

1.13 

.13 

.06 

6.07 

BBB 

0 

.84 

1.39 

.81 

0 

3.04 

BBB- 

0 

.99 

,94 

.48 

.19 

2.59 

BB+ 

0 

.08 

.19 

.35 

0 

.61 

BB 

0 

.14 

.04 

.66 

.19 

1.03 

BB- 

0 

0 

.14 

,23 

.08 

.44 

B+ 

0 

0 

0 

,03 

.06 

.09 

B 

0 

.01 

0 

.06 

.50 

.57 

B- 

0 

0 

0 

0 

.27 

.27 

Total 

25.21 

57.25 

11.57 

4.62 

1.34 

100.00 

Source:  GAO  analysis  of  data  supplied  by  Moody's,  Standard  &  Poor's,  FDIC,  the  Board  of  Governors  of 
the  Federal  Reserve  System,  and  OCC. 
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We  used  correlation  analysis  to  evaluate  the  question  of  relative  timing 
of  rating  changes — whether  nrsros  or  regulators  reported  changes  in 
risk  first.  Correlation  coefficients  were  computed  using  the  value  of 
NRSRO  ratings  at  a  particular  date  matched  with  camel  ratings  as  of  the 
same  date,  earlier  dates,  and  later  dates.  Our  expectation  was  that  if 
CAMEL  changed  assessments  of  risk  consistently  before  Moody’s  or  Stan¬ 
dard  &  Poor’s,  then  the  highest  correlation  coefficients  would  occur 
when  Moody’s  or  Standard  &  Poor’s  values  were  correlated  with  camel 
values  at  some  consistently  earlier  date. 

Figure  IV .  1  indicates  the  correlation  coefficients  between  camel  ratings 
and  Moody’s  ratings.  The  middle  bar  shows  the  correlation  coefficient 
when  the  series  were  matched  at  the  same  dates.  Bars  to  the  left  of  the 
middle  show  cases  when  Moody’s  ratings  were  matched  with  camel  rat¬ 
ings  of  earlier  dates  by  1  to  3  months.  The  bars  to  the  right  of  the  middle 
show  Moody’s  values  matched  with  camel  values  of  1  to  3  months  later. 
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Figure  IV. 1:  Correlations  of  IVloody’s  and 
CAf^EL  Ratings  Matched  at  Various 
Points  in  Time 


.60  Corroladon  CosffJcSsnt 

.55 


Note:  Monthly  data  from  December  1985  through  December  1990  were  used. 

Source:  GAO  analysis  based  on  data  supplied  by  Moody’s  Investors  Service  and  federal  bank 
regulators. 


The  figure  suggests  that  camel  may  have  reported  changes  in  risk  1 
month  before  Moody’s.  However,  other  factors  led  us  to  discount  this 
conclusion.  First,  the  correlation  coefficients  differed  only  slightly. 
Second,  there  was  a  discrepancy  in  the  way  regulators  and  nrsros  dated 
their  ratings  that  made  the  camel  ratings  appear  to  be  prematurely 
dated.  Regulators  dated  their  camel  assessment  either  the  date  the 
examination  began  or  the  date  of  the  financial  period  that  was  being 
examined.  Both  would  predate  the  assignment  of  a  camel  rating  by  an 
average  of  about  3  to  4  months,  according  to  regulatory  officials.  By 
contrast,  private  rating  agencies  used  the  date  that  their  assessment  was 
made  public.  Making  a  3-month  correction  for  the  premature  regulatory 
date  would  suggest  that  Moody’s  reported  changes  in  risk  2  months 
before  camel.'  However,  because  the  coefficients  still  differed  only 
slightly,  we  concluded  that  there  was  no  significant  lead  by  either 
Moody’s  or  camel. 


^Making  a  correction  on  fig.  IV.  1  would  amount  to  shifting  the  labels  on  the  bars  three  bars  to  the  left. 
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Figure  IV.2;  Correlations  of  Standard  & 
Poor’s  and  CAMEL  Ratings  Matched  at 
Various  Points  in  Time 


Figure  IV.2  shows  the  results  obtained  by  matching  camel  ratings  with 
Standard  &  Poor’s  ratings  at  various  dates.  Except  for  the  fact  that  the 
correlation  coefficients  are  lower,  we  otherwise  found  the  results  to  be 
comparable  to  the  Moody’s  case.  The  differences  between  bars  are  minor 
and  the  regulatory  dating  practices  falsely  suggest  that  camel  reported 
changes  in  risk  3  months  before  Standard  &  Poor’s. 


.60  Correlation  Coefficient 

.55 

.50 

.45 

.40 


Note:  Monthly  data  from  December  1985  through  December  1990  were  used. 

Source:  GAO  analysis  based  on  data  supplied  by  Standard  &  Poor’s  and  federal  bank  regulators. 


Page  25 


GAO/GOD-92-1 0  Government-Sponsored  Enterprises 


We  used  event  history  analysis'  to  examine  whether  either  camkl  ratings 
or  Moody’s  ratings  on  average  were  faster  in  reporting  changes  in  risk.- 
We  specifically  focused  on  risk  increases  because  that  was  the  case  most 
relevant  to  an  evaluation  of  a  regulatory  safe  harbor. 

To  determine  whether  Moody’s  ratings  or  G4MEL  ratings  were  faster  in 
reporting  increases  in  risk,  we  devised  a  three-step  test.  First,  we  chose 
a  correspondence  between  Moody’s  and  camel  rating  schemes  to  align 
the  13  risk  categories  used  by  Moody’s  with  camel’s  five  risk  categories. 
This  alignment  allowed  us  to  recognize  when  ratings  agreed  in  their 
assessments  of  risk. 

Second,  we  identified  instances  in  which  both  ratings  agreed  within  the 
same  period  that  risk  was  low  (this  instance  is  the  “original  level  of 
risk”)  and,  in  the  subsequent  period,  one  rating  indicated  a  one-level 
increase  in  risk  while  the  other  did  not.  We  computed  a  chi-square  sta¬ 
tistic  to  test  if  there  was  any  significant  difference  between  the 
probability  of  Moody’s  rating  reporting  increased  risk  with  the  camel 
rating  reporting  no  change  (condition  1),  and  the  camel  rating  reporting 
increased  risk  with  Moody’s  rating  reporting  no  change  (condition  2). 

In  the  third  step,  we  extended  our  investigation  of  the  condition,  if  any, 
that  we  found  significantly  more  probable.  If  both  ratings  agreed  in  a 
subsequent  period  that  risk  had  increased,  we  would  denote  the  rating 
that  changed  first  as  leading.  Otherwise,  the  results  would  be  considered 
indeterminate. 

We  aligned  the  rating  scales  and  did  the  analysis  several  ways,  allowing 
for  increasing  distinctions  between  levels  of  risk,  starting  with  high/low, 
then  with  high/medium/low  and  finally  with  high,  medium/high, 
medium/low,  and  low  risk.  For  brevity,  we  present  only  examples  from 
the  simplest  division  and  an  excerpt  from  a  table  with  the  finest 
division. 


^  This  analysis  assumes  that  an  event  history  model  called  the  discrete  time  Mark{.)v  chain  is  appro¬ 
priate.  Classical  expositions  of  this  model  are  in  Feller,  W.,  An  Introduction  to  Probability  Theory^  and 
Its  Applications,  3rd  ed.,  New  York:  John  Wiley,  1966;  and  Karlin,  S.  and  H.  Taylor,  A  First  Course  in 
Stochastic  processes,  New  York:  Academic  Press,  1975,  The  Markov  chain  model  assumes  that  the 
transition  probabilities  are  constant  in  time  and  among  banks.  If  these  assumptions  are  incorrect,  the 
dependence  on  past  histoiw^  is  more  complex,  or  exogenous  variables  attecting  the  process  have  been 
omitted,  this  model  would  be  wrong. 

^Because  of  time  constraints,  this  analysis  was  completed  for  Moody’s  data  only,  where  the  correla¬ 
tion  with  CA.MEL  data  was  higher. 
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The  test  represented  by  table  V.l  used  the  high/low  distinction  and  was 
one  of  seven  trials  run  using  this  risk  distinction.  In  this  test,  which 
involved  no  correction  for  the  premature  regulatory  date,  the  camel 
rating  seemed  to  lead. 

Table  V.l:  Frequencies  and  Associated  Probabilities  in  Moving  From  One  Risk  Level  to  Another  (Case  1:  CAMEL  Leads) 

Next  month 

Start  month 

Both  low 

Moody’s  high; 
CAMEL  low 

CAMEL  high; 
Moody’s  low 

Both  high 

Total 

Both  low 

5074 

19 

40 

1 

5134 

(.9883) 

(.0037) 

(.0078) 

(.0002) 

(1.0000) 

Moody’s  high; 

6 

283 

0 

11 

300 

CAMEL  low 

(.0200) 

(.9433) 

(.0000) 

(.0367) 

(1.0000) 

CAMEL  high; 

8 

0 

509 

31 

548 

Moody’s  low 

(.0146) 

(.0000) 

(.9288) 

(.0566) 

(1.0000) 

Both  high 

0 

4 

4 

866 

874 

(.0000) 

(.0046) 

(.0046) 

(.9908) 

(1.0000) 

Note:  Figures  are  for  the  case  of  CAMEL  low-risk  defined  to  be  CAMEL  1  or  2  and  high-risk  defined  as 
CAMEL  3,  4,  or  5;  Moody’s  low-risk  defined  to  be  ratings  from  Aaa  to  A2:  high-risk  defined  to  be  ratings 
from  A3  to  Ba3.  Probabilities  are  in  parentheses. 


Source;  GAO  analysis  based  on  data  supplied  by  Moody’s  Investors  Service  and  federal  bank 
regulators. 

The  following  paragraphs  summarize  the  statistical  analysis  for  this 
case.  Starting  from  the  condition  in  which  both  ratings  say  risk  is  low, 
we  tested  for  significant  difference  in  the  probability  of  moving  to  the 
condition  of  only  camel  reporting  higher  risk  (40  observations)  and  to 
the  condition  of  only  Moody’s  reporting  higher  risk  (19  observations). 
The  calculation  was  as  follows: 

Chl-square  statistic  (40  -  29.5y  +  (19  -  29.5y  ,  7  47 

2Ii7 .0 

The  chi-square  statistic  of  7.47  is  greater  than  6.64,  the  chi-square  sta¬ 
tistic  at  the  .99  level  of  significance  with  1  degree  of  freedom.  Therefore, 
the  CAMEL  rating  is  faster  in  saying  that  risk  has  increased. 

We  then  tested  whether  in  the  following  period  Moody’s  was  more  likely 
to  have  agreed  with  camel  that  risk  had  increased  or  whether  camel  was 
more  likely  to  return  to  the  less-risky  Moody’s  rating.  We  examined  the 
values  in  the  cells  that  denote  ratings  moving  from  “camel  high; 

Moody’s  low”  to  either  “Both  high”  (31  observations)  or  “Both  low”  (8 
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observations)  and  tested  whether  one  was  significantly  larger  than  the 
other.  Our  results  were  as  follows: 


Chi-square  statistic  = 


(8-  19.5)2 +  (31  _  19.5)2 
19.5 


13.56 


The  chi-square  of  13.56  is  greater  than  6.64,  the  chi-square  statistic  at 
the  .99  level  of  significance  with  1  degree  of  freedom.  Therefore,  the 
number  of  times  that  Moody’s  agreed  with  camel  in  the  next  period  that 
risk  had  increased  was  significantly  greater  than  the  number  of  times 
that  CAMEL  reverted  to  saying  that  risk  had  not  increased.  Since  they 
both  eventually  agreed  that  risk  had  increased,  we  can  say  that  the 
QAMEL  rating  was  faster  in  detecting  risk  increases. 


Table  V.2  examines  the  same  case  with  an  adjustment  for  the  premature 
regulatory  date.  After  a  correction  of  the  regulatory  date  by  3  months, 
the  results  became  indeterminate.^ 


- - - ,, - - - — - -TTT— - V-.' - . .  . . ,  . .  ........ 

- i — ■  ■  .  ■-.'L.  -.L'~  . -,vviL, — '  - - -  ■“  ^ 

Table  V.2:  Frequencies  and  Associated  Probabilities  in  Moving  From  One  Risk  Level  to  Another  With  a  3-fVlonth  Adjustment  for 
Premature  Regulatory  Date  (Case  2:  Indeterminate  Results) 

Start  month 

Next  month 

Total 

Both  low 

Moody’s  high; 
CAMEL  low 

CAMEL  high; 
Moody’s  low 

Both  high 

Both  low 

5138 

29 

25 

1 

5193 

(.9894) 

(.0056) 

(.0048) 

(.0002) 

(1 .0000) 

Moody's  high; 

5 

305 

0 

20 

330 

CAMEL  low 

(.0152) 

(.9242) 

(.0000) 

(.0606) 

(1,0000) 

CAMEL  high: 

8 

0 

445 

21 

474 

Moody’s  low 

(.0169) 

(,0000) 

(.9388) 

(.0443) 

(1,0000) 

Both  high 

1 

3 

4 

833 

841 

(.0012) 

(.0036) 

(.0048) 

(.9905) 

(1,0000) 

Note:  Figures  are  for  the  case  of  CAMEL  low-risk  defined  to  be  CAMEL  1  or  2  and  high-risk  defined  as 
CAMEL  3,  4,  or  5;  Moody's  low-risk  defined  to  be  ratings  from  Aaa  to  A2;  high-risk  defined  to  be  ratings 
from  A3  to  Ba3.  Probabilities  are  in  parentheses. 


Source:  GAO  analysis  based  on  data  supplied  by  Moody's  Investors  Service  and  federal  bank 
regulators. 


Starting  with  the  condition  of  both  Moody’s  and  camel  rating  risk  as 
low,  we  tested  first  for  significant  difference  in  the  probability  of 


^In  this  case,  the  results  would  have  remained  indetenninate  even  if  the  correction  for  the  premature 
regulatoO'  date  had  been  2  months. 


Page  28 


GAO/G'GD-92-10  Govenunent-Sponsored  Enterprises 


Appendix  V 

Event  History  Analysis 


moving  to  the  condition  of  only  camel  saying  risk  was  high  (26  observa¬ 
tions)  and  to  the  condition  of  only  Moody’s  rating  risk  as  high  (29  obser¬ 
vations).  Results  follow: 

Chi-square  statistic  =  ~  =  .30 

The  chi-square  of  .30  is  less  than  3.84,  the  chi-square  statistic  at  the  .95 
level  of  significance  with  1  degree  of  freedom.  Therefore,  we  could  not 
say  that  the  camel  rating  was  faster  in  detecting  increases  in  risk.  After 
we  made  a  3-month  adjustment  for  the  premature  regulatory  date,  the 
results  became  indeterminate.  This  would  have  also  been  the  case  if  we 
had  assumed  that  the  adjustment  was  2  months  instead  of  3. 

Table  V.3  presents  the  results  of  the  event  history  analysis  for  an  alter¬ 
native  association  of  Moody’s  ratings  with  camel  ratings.  In  this  case, 
the  results  were  indeterminate.  However,  when  an  adjustment  was  made 
for  the  premature  regulatory  date,  as  shown  in  table  V.4,  Moody’s  was 
shown  to  lead. 


Table  V.3:  Frequencies  and  Associated  Probabilities  in  Moving  From  One  Risk  Level  to  Another  (Case  3:  Indeterminate  Results) 

Start  month 

Next  month 

Total 

Both  low 

Moody’s  high; 
CAMEL  low 

CAMEL  high; 
Moody’s  low 

Both  high 

Both  low 

4530 

27 

29 

0 

4586 

(.9878) 

(.0059) 

(.0063) 

(.0000) 

(1.0000) 

Moody's  high; 

8 

817 

0 

23 

848 

CAMEL  low 

(.0094) 

(.9634) 

(.0000) 

(.0271) 

(1.0000) 

CAMEL  high; 

5 

0 

347 

25 

377 

Moody’s  low 

(.0133) 

(.0000) 

(.9204) 

(.0663) 

(1,0000) 

Both  high 

0 

7 

1 

1037 

1045 

(.0000) 

(.0067) 

(.0010) 

(.9923) 

(1.0000) 

Note:  Figures  are  for  the  case  of  CAMEL  low-risk  defined  to  be  CAMEL  1  or  2  and  high  risk  defined  as 
CAMEL  3,  4,  or  5;  Moody’s  low-risk  defined  to  be  ratings  from  Aaa  to  At ;  high-risk  defined  to  be  ratings 
from  A2  to  Ba3,  Probabilities  are  in  parentheses. 


Source:  GAO  analysis  based  on  data  supplied  by  Moody’s  Investors  Service  and  federal  bank 
regulators. 

Starting  with  the  condition  of  both  Moody’s  and  camel  rating  risk  as 
low,  we  tested  first  for  significant  difference  in  the  probability  of 
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moving  to  the  condition  of  only  camel  rating  risk  as  high  (29  observa¬ 
tions)  and  to  the  condition  of  only  Moody’s  rating  risk  as  high  (27  obser¬ 
vations).  Results  follow: 


Chi-square  statistic 


(29  -  28)2  -f  (27  -  28)2  _ 
28 


The  chi-square  of  .07  is  less  than  the  chi-square  statistic  of  3.84  at  the 
.95  level  of  significance  with  1  degree  of  freedom.  Therefore,  we  could 
not  say  that  camel  was  faster  in  indicating  increases  in  risk  because  the 
difference  between  27  and  29  was  not  large  enough.  However,  when  an 
adjustment  was  made  for  the  premature  regulatory  date,  the  results 
changed,  as  shown  in  table  V.4. 


Table  V.4:  Frequencies  and  Associated  Probabilities 
Premature  Regulatory  Date  (Case  4:  Moody’s  Leads) 

in  Moving  From  One  Risk  Level  to  Another  With  a  3-Month  Adjustment  for 

Start  month 

Next  month 

Total 

Both  low 

Moody’s  high; 
CAMEL  low 

CAMEL  high; 
Moody’s  low 

Both  high 

Both  low 

4572 

34 

18 

0 

4624 

(.9888) 

(.0074) 

(.0039) 

(.0000) 

(1.0000) 

Moody’s  high; 

7 

864 

0 

28 

899 

CAMEL  low 

(.0078) 

(.9611) 

(.0000) 

(.0311) 

(1.0000) 

CAMEL  high; 

5 

0 

301 

18 

324 

Moody’s  low 

(.0154) 

(.0000) 

(.9290) 

(.0556) 

(1.0000) 

Both  high 

1 

6 

1 

983 

991 

(.0010) 

(.0061) 

(.0001) 

(.9919) 

(1.0000) 

Note;  Figures  are  for  the  case  of  CAMEL  low-risk  defined  to  be  CAMEL  1  or  2  and  high-risk  defined  as 
CAMEL  3,  4,  or  5;  Moody's  low-risk  defined  to  be  ratings  from  Aaa  to  A1 ;  high-risk  defined  to  be  ratings 
from  A2  to  Ba3.  Probabilities  are  in  parentheses. 


Source:  GAO  analysis  based  on  data  supplied  by  Moody’s  Investors  Service  and  federal  bank 
regulators. 


Starting  with  the  condition  of  both  Moody’s  and  camel  rating  risk  as 
low,  we  tested  first  for  significant  difference  in  the  probability  of 
moving  to  the  condition  of  only  camel  rating  risk  as  high  (18  observa¬ 
tions)  and  to  the  condition  of  only  Moody’s  rating  risk  as  high  (34  obser¬ 
vations),  with  the  following  result: 


Chi-square  statistic 


(34  -  26)2  -f  (18  -  26)2  _ 
26 
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A  chi-square  of  4.92  is  greater  than  the  chi-square  statistic  of  3.84  at  the 
.95  level  of  significance  with  1  degree  of  freedom.  Therefore,  Moody’s 
was  faster  in  reporting  increased  risk. 

We  then  checked  whether  it  was  more  likely  for  the  camel  rating  to  have 
agreed  with  Moody’s  in  the  next  period  that  risk  had  increased  (28 
observations)  or  if  it  was  more  likely  that  Moody’s  would  have  changed 
back  to  saying  that  risk  had  not  increased  (7  observations).  The  result 
was  as  follows: 

Chi-square  statistic  =  g- 17.6y  +  (28- 17.5y  ,  ^ 

17.5 

The  12.60  chi-square  is  greater  than  the  chi-square  statistic  of  6.64  at 
the  .99  level  of  significance  with  1  degree  of  freedom.  Therefore,  it  was 
more  likely  that  camel  would  have  changed  to  agree  with  Moody’s  that 
risk  had  increased  than  Moody’s  would  have  changed  to  agree  with 
CAMEL  that  risk  had  not  increased.  As  a  result,  for  this  case,  when  an 
adjustment  was  made  for  the  premature  regulatory  date,  we  could  say 
that  Moody’s  reported  increases  in  risk  before  camel. 

1 

I 

[ 

We  did  the  analysis  several  additional  ways.  We  tried  a  three-way  divi¬ 
sion  of  low-,  medium-,  and  high-risk;  and  a  four-way  division  with  cate¬ 
gories  comparable  to  camel  1,  2,  3,  and  4-5 — the  finest  division  of  risk 
categories  we  analyzed. 

When  the  finest-risk  breakdown  possible  was  examined,  neither  camel 
nor  Moody’s  led  in  detecting  increases  in  risk  at  the  least-risky  end  of 
the  ratings  scale.  This  result  held  whether  or  not  an  adjustment  was 
made  for  the  premature  regulatory  date. 

Table  V.5  illustrates  one  of  the  cases  in  which  we  divided  risk  into  four 
categories  corresponding  to  camel  1,  2,  3,  and  4-5.  For  this  example,  we 
divided  Moody’s  ratings  using  a  frequency  distribution  as  close  as  pos¬ 
sible  to  the  CAMEL  frequency  distribution.  As  a  result,  Moody’s  ratings 
from  Aaa  to  Aa2  were  assumed  to  correspond  to  camel  1  and  those  from 
Aa3  to  A2  were  assumed  to  correspond  to  camel  2. 

Since  the  complete  table  would  contain  a  total  of  256  cells — 16  possible 
conditions  for  the  start  month  and  16  possible  conditions  for  the  next 
month,  we  presented  only  the  data  relevant  for  computing  the  chi- 
square  statistics  needed  for  our  tests.  As  a  result,  the  total  number  of 
observations  in  the  right  column  is  greater  than  the  sum  of  observations 
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across  the  four  columns  containing  the  cases  we  highlighted.  The  differ¬ 
ence  is  from  the  columns  containing  the  cases  we  did  not  highlight.  The 
same  is  true  for  the  probabilities. 


Table  V.5:  Frequencies  and  Associated  Probabilities  in  Moving  From  One  Risk  Level  to  Another  (Case  5:  Indeterminate  Results) 


Next  month 


Start  month 

Both  low 

Moody’s  higher; 
CAMEL  low 

CAMEL  higher; 
Moody’s  low 

Both  higher 

Total 

Both  low 

812 

11 

13 

0 

836 

(.9713) 

(.0132) 

(.0156) 

(.0000) 

(1.0000) 

Moody’s  higher; 

6 

506 

0 

13 

530 

CAMEL  low 

(.0113) 

(.9547) 

(.0000) 

(.0245) 

(1 .0000) 

CAMEL  higher: 

7 

0 

992 

28 

1030 

Moody’s  low 

(.0068) 

(.0000) 

(.9631) 

(.0272) 

(1.0000) 

Both  higher 

0 

7 

6 

2673 

2738 

(.0000) 

(.0026) 

(.0022) 

(.9763) 

(1.0000) 

Note  1 :  Figures  are  for  the  case  of  CAMEL  low-risk  defined  to  be  CAMEL  1  and  higher-risk  defined  as 
CAMEL  2;  Moody’s  lowest-risk  defined  to  be  ratings  from  Aaa  to  Aa2  and  higher-risk  defined  to  be 
ratings  from  Aa3  to  A2.  Probabilities  are  in  parentheses. 

Note  2:  The  total  number  of  observations  and  probabilities  in  the  right  column  is  greater  than  the  sum  of 
observations  across  the  columns  because  only  data  relevant  for  our  tests  are  given, 

Source:  GAO  analysis  based  on  data  supplied  by  Moody's  Investors  Service  and  federal  bank 
regulators. 

Starting  with  the  condition  of  both  Moody’s  and  camel  rating  risk  as 
low,  we  tested  first  for  significant  difference  in  the  probability  of 
moving  to  the  condition  of  only  camel  rating  risk  as  higher  (13  observa¬ 
tions)  and  to  the  condition  of  only  Moody’s  rating  risk  as  higher  (11 
observations).  The  result  was  as  follows: 


Chi-square  statistic 


(13  -  12)^ -t  (11 -12)^  =  IS 
11 


The  .18  chi-square  is  less  than  the  chi-square  statistic  of  3.84  at  the  .95 
level  of  significance  with  1  degree  of  freedom.  Therefore,  we  could  not 
say  that  camel  was  faster  than  Moody’s  in  indicating  increases  in  risk. 
The  result  was  not  changed  by  an  adjustment  for  the  premature  regula¬ 
tory  date,  nor  by  alternate  associations  between  camel  and  Moody’s 
ratings. 

The  results  varied  for  other  analyses  using  cruder  breakdowns  of  risk. 
For  low-  and  medium-risk  breakdowns,  when  an  adjustment  was  made 
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for  the  premature  regulatory  date,  some  associations  of  the  ratings 
resulted  in  Moody’s  reporting  changes  earlier,  while  other  associations 
gave  indeterminate  results.  In  no  instance  did  the  camel  rating  report 
changes  in  risk  first.  Only  if  no  correction  had  been  made  for  the  prema¬ 
ture  regulatory  reporting  date  would  camel  ratings  have  sometimes 
reported  increases  in  risk  first. 
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